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REWORD 


IS& 


Because  computer-based  simulations  are  safer  and  usually 
less  expensive  than  training  with  weapons  systems,  they  will  play 
an  increasingly  important  role  in  military  training. 

The  U.S.  Army  Research  Institute  for  the  Behavioral  and 
Social  Sciences  (ARI) ,  in  coordination  with  the  U.S.  Army  Project 
Manager  for  Training  Devices  (PM  TRADE)  and  the  University  of 
Central  Florida's  Institute  for  Simulation  and  Training  (1ST) , 
recently  began  a  research  program  to  examine  applications  of 
learning  and  training  principles  to  the  design  and  use  of  train¬ 
ing  devices.  The  research  program  was  established  to  identify 
conditions  under  which  alternatives  to  traditional  methods  for 
teaching  psychomotor  skills  such  as  tank  gunnery  can  yield  in¬ 
creased  trainee  proficiency,  decreased  training  costs,  or  both. 

The  research  reported  here  was  performed  by  the  Orlando 
Field  Unit  at  PM  TRADE  as  part  of  Task  3104,  "Advanced  Technology 
for  the  Design  of  Training  Devices."  The  research  program  ad¬ 
dresses  the  cost  effectiveness  of  training  devices  as  a  function 
of  the  fidelity,  instructional  features,  and  instructional  strat¬ 
egies.  PM  TRADE  sponsored  the  research,  which  was  accomplished 
under  a  Memorandum  of  Agreement  with  ARI  dated  24  October  1991. 
The  results  of  this  research  have  been  briefed  to  PM  TRADE'S 
Project  Manager,  Close  Combat  Training  Systems;  PM  TRADE'S 
Project  Manager,  Videodisc  Gunnery  Simulator  (VIGS) ;  PM  TRADE'S 
Research  and  Engineering  Management  Branch;  and  1ST.  Information 
about  the  conduct  and  results  of  the  experiment  was  also  provided 
to  the  manufacturer  of  VIGS. 


EDGAR  M.  JOHNSON 
Technical  Director 
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THE  EFFECTS  OF  DIFFERENCES  BETWEEN  PRACTICE  AND  TEST  CRITERIA  ON 
TRANSFER  AND  RETENTION  OF  A  SIMULATED  TANK  GUNNERY  TASK 


EXECUTIVE  SUMMARY 


Requirement : 

Most  training  devices  incorporate  an  instructional  subsystem 
that  controls  learning  and  retention  of  the  tasks  and  skills  the 
device  is  designed  to  train.  Instructional  subsystems  control 
such  aspects  of  the  training  as  difficulty  of  training  and  test 
criteria,  training  sequencing,  and  feedback  content  and  amount. 
The  instructional  subsystem  should  be  supported  by  research  find¬ 
ings  that  document  the  efficiency  of  the  approach  to  produce  de¬ 
sired  training  outcomes.  This  research  compares  the  effects  of 
practice  using  performance  criteria  that  were  easier  than,  more 
difficult  than,  or  the  same  as  the  test  and  retention  criterion. 


Procedure : 

Each  member  of  three  groups  of  15  randomly  assigned  male 
undergraduates  from  the  University  of  Central  Florida  received 
three  blocks  of  18  practice  trials  with  a  tank-gunnery  training 
device,  the  Videodisc  Gunnery  Simulator  (VIGS) .  VIGS  has  targets 
whose  kill  zone,  the  percentage  of  the  target-silhouette  area 
that  is  scored  as  a  kill  when  hit,  can  be  varied.  Each  of  the 
three  groups  practiced  with  the  kill  zone  at  one  of  three  set¬ 
tings:  50%,  100%,  or  150%  of  the  target-silhouette  area.  Sub¬ 

jects  in  each  groups  were  tested  immediately  after  training  with 
the  kill  zone  set  at  100%  and  were  tested  again  after  a  10-week 
retention  interval.  Dependent  variables  were  target  kills,  aim¬ 
ing  error,  and  time  to  fire. 


Findings : 

The  results  of  this  experiment  suggest  a  speed/ accuracy 
trade-off:  Practice  with  an  accuracy  criterion  more  difficult 
than  the  test  criterion  yielded  greater  kill  percentages  and 
slower  firing  times  than  did  practice  and  testing  with  the  same 
criterion.  This  trend,  although  not  statistically  significant, 
held  for  transfer-test  scores  and  retention-test  scores.  The 
effects  of  practicing  with  easier  criteria  than  the  test  criteria 
were  not  reliably  different  from  the  effects  of  practicing  with 
accuracy  criteria  that  were  the  same  as  the  test  criteria. 


•  • 
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Problems  with  the  test  scenarios  and  the  small  number  of  subjects 
available  for  retention  testing  contributed  to  the  unreliability 
of  differences  among  the  groups*  scores. 


Utilization  of  Findings: 

Training  system  developers  and  managers  should  consider  the 
results  of  this  and  related  experiments  in  deciding  whether  to 
use  reduced  kill  zones  and  other  training  methods  that  may  in¬ 
crease  the  accuracy  of  trainee  performance  at  the  expense  of 
speed.  The  aspects  of  tank-gunnery  training  doctrine  that  per¬ 
tain  to  the  relative  importance  of  speed  and  accuracy — the 
emphasis  on  first-round  hits,  for  example — should  be  reexamined 
in  light  of  literature  about  speed/ accuracy  trade-offs.  Addi¬ 
tional  research  should  aim  to  establish  exact,  quantified,  and 
generalizable  speed/accuracy  trade-offs  for  various  firing  con¬ 
ditions  in  simulators  and  in  weapons  systems. 
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THE  EFFECTS  OF  DIFFERENCES  BETWEEN  PRACTICE  AND  TEST  CRITERIA  ON 
TRANSFER  AND  RETENTION  OF  A  SIMULATED  TANK  GUNNERY  TASK 


Introduction 

Most  training  devices  incorporate  instructional  subsystems 
that  consist  of  instructional  features  such  as  freeze,  replay, 
and  performance  measurement  as  well  as  an  instructional  strategy 
that  includes  how  these  features  are  to  be  used  and  what  course 
the  training  is  to  take.  The  domain  of  instructional  features 
was  outlined  by  Hughes  (1979).  Sticha,  Singer,  Blacksten, 
Morrison,  and  Cross  (1990)  developed  a  prototype  expert  system 
that  incorporates  rules  for  the  selection  of  certain 
instructional  features.  Other  than  the  work  by  Sticha  et  al. 
(1990),  there  is  little  guidance  on  the  conditions  under  which  to 
use  various  instructional  features.  Perhaps  more  important, 
there  is  little  guidance  about  how  to  develop  an  instructional 
strategy  using  the  instructional  features.  Such  guidance  is 
necessary  because  one  can  imagine  conditions  under  which 
different  instructional  strategies  could  facilitate,  degrade,  or 
have  no  effect  on  learning  and/or  skill  retention. 

An  instructional  feature  of  three  of  the  U.S.  Army's  tank- 
gunnery  training  devices,  TOPGUN,  the  Videodisc  Gunnery  Simulator 
(VIGS) ,  and  the  Conduct  of  Fire  Trainer  (COFT) ,  is  the  capability 
to  vary  the  target  kill  zone.  The  kill  zone  is  the  percentage  of 
the  target-silhouette  area  that  is  scored  a  kill  when  hit.  Kill 
zones  can  be  set  to  be  the  same  size  as,  or  smaller  than,  the 
target  silhouette.  (The  Videodisc  Gunnery  Simulator  (VIGS) 
allows  the  kill  zone  size  to  be  larger  than  the  target 
silhouette.  Scoring  algorithms  award  points  for  rounds  striking 
within  the  programmed  kill  zone.  One  purpose  of  a  programmable 
kill  zone  is  to  teach  the  concept  that,  under  most  conditions,  a 
shot  that  hits  the  center  of  the  target  is  superior  to  shots  that 
hit  elsewhere. 

The  programmable  kill  zone  can  also  be  manipulated  within  an 
instructional  strategy.  The  evaluator/ instructor  can  aid  or 
challenge  the  student  gunner  by  increasing  or  decreasing  the  area 
in  which  rounds  must  hit  to  constitute  a  "kill"  (Ml/MlAl,  Tank 
Videodisc  Gunnery  Simulator  (VIGS),  Device  17-142,  Instructor's 
Utilization  Handbook  for  Simulation  Eguipment,  1988) .  The 
implication  seems  to  be  that  instructors  should  increase  the  size 
of  kill  zones  to  aid  students  who  are  having  difficulty  hitting 
targets  and  should  decrease  kill  zones  to  challenge  students  who 
are  hitting  targets  regularly. 

The  availability  of  programmable  kill  zones  relates  to  an 
unresolved  behavioral  science  issue  of  long  standing;  namely, 
under  what  conditions  may  learning  be  facilitated  (or  degraded) 
by  practice  in  which  criteria  are  more  difficult  than,  the  same 
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as,  or  less  difficult  than  the  criteria  used  in  tests  of  transfer 
and  retention?  Research  and  thinking  aimed  at  resolving  that 
issue  include  four  areas  of  inquiry: 

1.  Presumed  benefits  of  differences  between  practice 
criteria  and  test  criteria. 

2.  Asymmetrical  transfer  research. 

3.  Adaptive  training  research. 

4.  Dimensions  of  training  effectiveness. 

Presumed  benefits  of  differences  between  practice  criteria  and 
test  criteria.  A  considerable  amount  of  research  has  been 
conducted  to  examine  the  proposition  that  superior  transfer  of 
training  may  be  obtained  by  varying  practice  criteria  from 
transfer  criteria  (Mane  et  al.  1989).  Lintern  (1991)  noted  that 
advantages  for  both  practice  with  easier  criteria  and  for 
practice  with  more  difficult  criteria  have  been  hypothesized. 

Wightman  and  Lintern  (1985)  reviewed  strategies  for  part-task 
training  of  tracking  and  control  tasks.  They  recounted  assumed 
advantages  for  having  training  task  criteria  differ  from  transfer 
task  criteria.  An  argument  in  favor  of  training  with  an  easier 
criterion  is  that  for  training  very  difficult  tasks’,  initial 
practice  with  a  high  level  of  difficulty  may  preclude  meaningful 
practice.  Therefore,  initial  training  with  an  easy  version  of 
the  task  would  be  desirable.  An  argument  for  training  with  a 
more  difficult  criterion  is  that  practice  with  a  more  difficult 
version  of  the  task  forces  trainees  to  extend  themselves.  This 
may  lead  to  faster  learning  of  skills  needed  for  performance  of 
the  transfer  task.  Also,  practice  with  criteria  more  difficult 
than  transfer  criteria  may  establish  resistance  to  forgetting  and 
stress . 

Wightman  and  Lintern  pointed  out  that  using  a  practice  task 
that  is  less  difficult  than  the  transfer  task  is  common  in 
educational  and  instructional  environments.  However,  they  noted 
that  for  training  tracking  skills  for  manual  control  there  is 
little  empirical  evidence  to  indicate  that  training  with  an 
easier  criterion  is  more  effective  then  training  with  the  task 
criterion.  Nevertheless,  they  proposed  that  there  is  some 
indication  that  training  for  a  highly  difficult  transfer  task  may 
best  be  accomplished  by  training  with  a  moderately  difficult 
version  of  the  task.  They  recommended  further  research  in  this 
area. 

Wightman  and  Lintern  described  an  experiment  that  found 
superior  transfer  for  training  with  a  more  difficult  version  of 
the  transfer  task  than  training  with  the  transfer  task.  In  that 
experiment,  Williges  and  Baron  (1973)  employed  a  response  surface 
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methodology  to  measure  transfer  of  a  tracking  task  as  a  function 
of  three  variables;  number  of  training  trials,  time  between 
training  trials,  and  tracking  speed  of  the  training  task.  The 
measure  of  transfer  was  number  of  trials  on  the  transfer  task  to 
reach  a  time-on-target  criterion.  Practice  with  a  tracking  speed 
faster  than  the  speed  used  for  the  transfer  tracking  task  was 
found  to  produce  superior  transfer  than  practice  with  the 
tracking  speed  of  the  transfer  task. 

Ecklund  (1975)  cited  three  experiments  that  indicated  that 
training  with  practice  targets  that  were  smaller,  that  is  more 
difficult,  than  the  transfer  test  targets  produced  transfer 
superior  to  training  with  targets  of  the  size  of  the  test 
targets.  The  experiments  cited  by  Ecklund  were  conducted  by  Mace 
(1931)  and  Day  (1954) ,  both  employing  a  pencil  spearing  task,  and 
by  Igbanugo  (1972)  employing  a  basketball  passing  task.  Ecklund 
also  cited  an  experiment,  employing  throwing  tasks,  by  Scannel 
(1964)  that  involved  similar  training  manipulations  but  showed  no 
differences  in  training  effectiveness.  Ecklund  stated  that 
research  in  this  area  was  inconclusive.  He  noted  that  reducing 
the  size  of  basketball  rims  was  a  current  training  practice.  The 
practice  was  based  on  the  assumption  that  training  with  a  smaller 
rim  would  produce  better  shooting  accuracy  when  transferred  to  a 
regulation-size  rim  than  training  with  a  regulation-size  rim. 

Abel  (1986)  conducted  a  series  of  experiments  to  examine 
practice  effects  on  the  BATTLESIGHT  tank  gunnery  trainer. 
(BATTLESIGHT  was  a  prototype  for  the  TOPGUN  gunnery  trainer.) 

One  of  the  experiments  included  a  manipulation  to  investigate 
practice  with  a  reduced  kill  zone. 

Abel's  experiment  also  included  a  manipulation  to  look  at 
aspects  of  game  format,  for  example,  the  distribution  of 
ammunition  and  player  "lives".  There  was  no  significant  effect 
for  game  format  nor  was  there  a  significant  interaction  of  game 
format  and  kill  zone  size.  However,  the  variations  in  game 
format,  although  completely  crossed  with  kill  zone  size 
variation,  complicate  the  interpretation  of  the  effects  of  kill 
zone  size.  For  example,  the  game  format  manipulations  allowed, 
and  resulted  in,  considerable  variation  across  subjects  of  the 
number  of  rounds  of  ammunition  fired. 

Abel  found  that  subjects  firing  at  targets  with  reduced  kill 
zones,  in  contrast  to  subjects  firing  at  targets  with  100%  kill 
zones,  had  significantly  lower  percent  hits  and  percent  first 
round  hits.  (A  round  was  scored  as  a  "hit"  if  it  impacted  within 
the  defined  kill  zone.  Rounds  that  impacted  within  the  target 
silhouette,  but  outside  the  defined  kill  zone,  were  treated  by 
the  BATTLESIGHT  scoring  system  as  misses.)  Kill  zone  setting 
did  not  result  in  significant  differences  in  average  time  to 
fire.  For  both  levels  of  kill  zone  size,  time  to  fire  decreased 
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as  a  function  of  practice,  however,  hit  (kill)  rate  did  not 
significantly  improve  as  a  function  of  amount  of  practice. 


Abel  interpreted  the  improvement  in  firing  time  and  the  lack 
of  improvement  in  hit  rate  as  an  indication  of  subjects'  emphasis 
on  speed  over  accuracy.  Abel  presented  the  argument  that  many 
aspects  of  video  games,  especially  the  scoring  algorithms, 
encourage  emphasis  on  speed  rather  than  accuracy  of  performance. 

Abel  employed  hit  rate  as  a  measure  of  firing  accuracy. 
Stating  that  the  subjects  in  the  reduced  kill  zone  group  were 
less  accurate  than  the  subjects  in  the  standard  kill  zone  group 
raises  questions  of  terminology.  The  difference  in  group  scores 
was  probably  a  function  of  an  application  of  different  standards, 
not  a  function  of  differences  in  aiming  skill.  Measures  of 
aiming  error  deviation,  in  elevation  and  azimuth,  were  not 
available.  The  group  of  subjects  that  practiced  with  the  reduced 
kill  zone  was  not  transferred  to  firing  with  a  standard  kill 
zone. 

The  experiment  described  in  this  report  employed  the 
programmable  kill  zone  size  of  the  VIGS  trainer  to  examine  the 
instructional  strategy  of  varying  the  difficulty  of  practice 
criteria  in  relation  to  test  criteria.  This  research  differs 
from  that  conducted  by  Abel  in  that  the  focus  of  the  present 
experiment  is  on  transfer  and  retention,  not  only  on  differences 
in  performance  during  practice  itself. 

Asymmetrical  transfer  research.  In  the  asymmetrical  transfer 
paradigm  one  group  practices  with  an  easy  version  of  a  task  and 
then  the  group  is  tested  with  a  difficult  version  of  the  task. 

The  amount  of  transfer  is  compared  with  that  of  a  group  for  which 
initial  practice  is  with  the  difficult  version  of  the  task  and 
transfer  is  measured  for  performance  of  the  easy  version  of  the 
task.  Day  (1956)  reviewed  several  experiments  that  demonstrated 
asymmetrical  transfer.  Some  of  these  experiments  demonstrated 
that  easy-to-diff icult  sequencing  of  practice  and  transfer  tasks 
results  in  higher  transfer  than  a  diff icult-to-easy  sequence. 
Other  experiments  have  found  better  transfer  with  the  difficult- 
to-easy  sequence.  Day  pointed  out  that  these  findings  of 
asymmetrical  transfer  have  important  implications  for  practical 
issues  in  training  as  well  as  for  transfer  theory. 

Day  proposed  that  a  skilled  task  possesses  three  basic 
features:  a  stimulus  complex  (display),  operator  control  devices, 
and  the  linkage  between  display  and  controls.  The  difficulty  of 
a  task  may  vary  as  a  function  of  changes  in  one  or  more  of  these 
features.  According  to  Day,  previous  research  indicated  that 
transfer  was  greater  for  easy-to~diff icult  when  stimulus  alone 
was  varied.  Transfer  was  greater  for  diff icult-to-easy  when  task 
difficulty  was  varied  with  respect  to  response  variables,  for 
example,  rate  of  responding.  However,  Holding  (1965)  noted  that 
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manipulations  of  task  difficulty  often  involve  changes  in  both 
the  stimulus  and  response  components  of  the  task. 

Day  emphasized  that  many  of  these  experiments  were  designed 
without  control  groups.  That  is  there  were  no  easy-to-easy  or 
dif f icult-to-dif f icult  comparison  groups.  He  pointed  out  that 
such  groups  are  important,  for  both  practical  and  theoretical 
reasons,  to  allow  conclusions  about  maximum  transfer.  He  noted 
inconsistencies  in  this  area  of  investigation  in  defining 
difficulty,  scoring,  and  estimating  extent  of  transfer.  These 
inconsistencies  hamper  explanation  of  the  differential  findings. 

Holding  (1962)  reviewed  more  than  thirty  studies  concerned 
with  the  transfer  of  training  between  difficult  and  easy  tasks. 
Although  many  of  the  studies  demonstrated  asymmetrical  transfer, 
neither  order  of  training,  easy-to-diff icult  nor  dif ficult-to- 
easy,  was  consistently  superior.  He  pointed  out  that  this 
inconsistency  of  superiority  for  one  approach  over  the  other 
occurred  not  only  across  kinds  of  tasks  but  could  also  be 
expected  for  different  stages  of  learning  for  individual  tasks. 
That  is,  with  small  amounts  of  practice,  easy-to-diff icult 
transfer  may  be  superior  but  for  transfer  after  larger  amounts  of 
practice  dif ficult-to-easy  transfer  may  be  greater. 

Holding  (1965)  proposed  that  the  concept  of  inclusion  may 
provide  a  partial  explanation  of  the  instances  for  which 
dif ficult-to-easy  produces  greater  transfer  than  the  reverse. 

Dif ficult-to-easy  transfer  is  superior  because  the  difficult  task 
includes  components  of  the  easy  task.  He  provided  examples  of 
inclusion:  after  practicing  to  shoot  an  apple  no  further  practice 
is  needed  to  hit  a  barn  door,  a  telegraphic  operator  who  can 
receive  Morse  code  sent  at  twenty-four  words  a  minute  needs  no 
additional  training  to  receive  code  at  a  slower,  easier,  rate. 

Holding  (1962)  suggested  that  instances  of  superiority  of 
easy-to-dif ficult  transfer  conditions  could  be  explained  in  terms 
of  error  size  constancy.  Initial  practice  with  the  easier  task 
leads  to  smaller  error  size  than  practice  with  the  difficult 
task.  Holding  proposed  that  there  is  a  tendency  to  carry  over 
the  mean  error  size  from  the  practice  task  to  the  transfer  task. 
Holding  (1965)  restated  this  explanation  of  findings  of 
superiority  of  easy-to-diff icult  transfer,  using  the  term 
"performance  standards".  (Performance  standards  referred  to  the 
level  of  performance  of  the  subject,  not  task  criteria  set  by  the 
experimenter.)  High  performance  standards  are  established  under 
the  easy  condition.  Looser  performance  standards  are  established 
under  the  difficult  condition.  Under  the  transfer  condition, 
subjects  tend  to  maintain  the  standards  established  during 
practice.  Holding  noted  that  if  the  difficult  condition  were 
much  more  difficult  than  the  easy  condition  then  performance 
standards  could  not  be  maintained.  For  this  situation  superior 
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dif f icult-to-easy  transfer  would  be  observed  as  a  function  of 
inclusion. 

Holding  (1962)  stated  that  the  interaction  between  inclusion 
and  error  constancy  is  further  complicated  by  the  strategies 
adopted  by  subjects  and  by  the  emphasis  on  speed  or  accuracy 
demanded  by  the  structure  of  the  task.  Holding  maintained  that 
there  is  an  extremely  complex  interaction  of  factors  that 
influence  transfer  between  difficult  and  easy  tasks.  In 
addition,  he  emphasized  that  findings  of  asymmetrical  transfer  do 
not  indicate  that  practice  with  an  easier  or  more  difficult 
version  of  the  task  is  a  better  training  approach  than  practice 
with  the  transfer  task. 

Holding  (1962,  1965)  described  general  problems  with  using 
the  concept  of  task  difficulty  in  predicting  transfer  of 
training.  Holding  (1962)  pointed  out  that  the  terms  "easy"  and 
"difficult"  are  used  in  a  relative  sense  within  experiments.  The 
condition  designated  as  "difficult"  in  one  experiment  might 
actually  be  easier  than  the  "easy"  condition  of  another 
experiment.  Reconciliation  of  conflicting  results  across 
experiments  is  hampered  by  the  lack  of  a  measure  of  absolute 
difficulty. 

Holding  (1965)  noted  that  the  term  "difficult"  is  used  to 
describe  tasks  that  show  high  error  scores  or  long  learning 
curves.  Many  different  task  characteristics,  such  as  speed  or 
target  size,  contribute  to  task  difficulty.  Each  of  these 
characteristics  may  lead  to  different  transfer  effects.  Holding 
argued  that  it  may  be  more  useful  to  concentrate  on  task 
characteristics,  rather  than  task  difficulty,  in  predicting 
transfer  effects. 

Adaptive  training  research.  Kelley  (1969)  defined  adaptive 
training  as  training  in  which  the  practice  task  is  varied  as  a 
function  of  how  well  the  trainee  performs.  With  adaptive 
training  the  difficulty  of  the  practice  task  is  adjusted  to  the 
performance  level  of  the  trainee.  Task  difficulty  is  increased 
after  successful  trials  and  decreased  after  unsuccessful  trials. 
To  support  adaptive  training  three  features  are  required  of 
training  simulators.  1)  The  simulator  must  have  a  performance 
measurement  system  capable  of  continuously  or  repetitively 
measuring  trainee  performance.  2)  A  means  of  adjusting  some 
characteristic,  the  adaptive  variable,  of  the  task  is  required. 

3)  An  adaptive  logic  which  automatically  changes  the  adaptive 
variable  as  a  function  of  performance  is  needed. 

Adams  (1987)  pointed  out  that  the  adaptive  training  approach 
is  based  on  the  assumption  that  training  on  the  transfer  task 
itself  is  not  necessarily  the  most  efficient  approach  to 
training.  In  comparison  to  practice  on  the  criterion  task 
itself,  the  adaptive  sequence  of  practice  presumably  leads  to 
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faster  learning,  or  a  higher  level  of  learning,  of  the  criterion 
task.  Adams  stated  that  adaptive  training  is  an  intuitively 
attractive  idea  but  has  failed  to  prove  effective  for  training 
tracking  tasks. 

More  recently,  Lintern  (1991)  noted  that  although  early 
attempts  to  demonstrate  an  advantage  for  adaptive  training  were 
unsuccessful,  recent  attempts  have  been  more  encouraging.  He 
cited  research  by  Mane  et  al.  (1989)  that  employed  as  the 
adaptive  variable  the  speed  of  hostile  forces  in  a  video  game. 
Practice  with  a  reduced  speed,  half  criterion  speed,  was  superior 
to  control  training.  However,  practice  with  a  more  greatly 
reduced  speed,  one-quarter  criterion,  was  not  advantageous.  He 
proposed  that  two  competing  processes  may  come  into  play  when  a 
task  is  simplified  during  adaptive  training.  Simplification  of  a 
task  during  practice  may  aid  skill  acquisition  but  may  also 
distort  the  task,  reducing  positive  transfer.  To  be  effective  an 
adaptive  training  manipulation  must  take  advantage  of  the 
benefits  of  simplification  while  minimizing  the  disadvantages. 

Dimensions  of  training  effectiveness.  Mane,  Adams,  and 
Donchin  (1989)  emphasized  that  transfer  of  training  is  a  key 
criterion  to  evaluating  the  usefulness  of  a  training  program. 
Level  of  performance  on  the  practice  task  is  not  necessarily 
predictive  of  the  amount  of  transfer  that  will  take  place. 
Reducing  the  difficulty  level  of  the  practice  task  will  produce 
better  performance  during  practice  but  may  result  in  less 
transfer  to  the  target  task. 

Schmidt  and  Bjork  (in  preparation)  argued  that  a 
comprehensive  measure  of  the  effectiveness  of  military  training 
must  not  be  based  entirely  on  performance  during  training  or  on  a 
short-term  transfer  test.  Rather,  measurement  of  training 
effectiveness  must  place  emphasis  on  measures  of  retention. 
Retention  is  the  critical  factor  for  ensuring  combat  readiness 
during  emergencies  and  for  minimizing  retraining  time.  They 
noted  that  generalization  and  capability  to  perform  under  stress 
should  also  be  considered  in  evaluating  the  effectiveness  of 
military  training. 

They  contended  that  current  training  designs  are  often  based 
on  an  erroneous  assumption:  variations  in  practice  conditions 
that  result  in  the  highest  level  of  performance  during  practice 
represent  the  best  training  approach.  Schmidt  and  Bjork 
described  three  experimental  manipulations  of  practice  conditions 
that,  relative  to  control  conditions,  led  to  slower  acquisition 
or  inferior  performance  during  practice  but  better  performance  on 
tests  of  retention.  The  manipulations  involved  exercise 
sequence,  feedback  schedule,  and  task  variation.  These  findings 
lead  us  to  question  the  effectiveness  of  current  training 
approaches  and  suggest  alternatives  that  may  lead  to  low-cost 
methods  of  improving  simulator-based  training.  In  addition,  the 
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findings  indicated  that  advantages  of  variations  in  practice 
conditions  that  lead  to  superior  retention  may  go  undetected  by 
transfer  tests  conducted  soon  after  practice. 

The  practice  manipulations  used  in  this  experiment  differ 
from  those  cited  by  Schmidt  and  Bjork.  However,  their  argument 
that  evaluations  of  the  training  effectiveness  of  practice 
manipulations  should  include  measures  of  retention  was  one  of  the 
factors  which  led  us  to  take  a  new  look  at  an  old  issue:  the 
optimal  relation  between  practice  criteria  and  (transfer)  test 
criteria. 

Purpose.  Rationale,  and  Hypotheses.  There  are  theoretical 
arguments,  and  some  equivocal  empirical  evidence,  that  practice 
with  criteria  that  differ  from  test  criteria  can  produce  training 
superior  to  training  for  which  practice  and  test  criteria  are  of 
equivalent  difficulty.  Arguments  for  practice  with  easier 
criteria  and  arguments  for  practice  with  more  difficult  criteria 
have  been  presented.  Wells  and  Hagman  (1989)  pointed  out  that 
research  to  date  had  not  provided  practical  guidelines  for 
determining  when  one  approach  is  to  be  preferred  over  the  other. 

The  U.S.  Army  Research  Institute,  under  sponsorship  by  the 
U.S.  Army  Project  Manager  for  Training  Devices,  and  in 
coordination  with  the  University  of  Central  Florida's  Institute 
for  Simulation  and  Training,  has  begun  a  research  program  to 
examine  potential  advantages  of  alternatives  to  traditional 
training  methods.  This  report  describes  the  first  experiment 
conducted  under  that  program.  The  purpose  of  this  experiment  was 
to  evaluate  manipulation  of  the  difficulty  level  of  practice 
criteria  as  a  method  of  improving  transfer  and  retention  of  a 
track-and-shoot  task. 

Two  competing  hypotheses  were  considered.  The  first  was  that 
practice  with  a  more  difficult  criterion,  the  kill  zone  set  to 
50%  of  the  size  of  the  target  silhouette,  would  lead  to  better 
transfer  and  retention  than  practice  with  the  transfer  (control) 
criterion  itself,  the  kill  zone  set  to  100%  of  the  size  of  the 
target.  We  hypothesized  that  practice  with  the  more  difficult 
accuracy  criterion  would  lead  to  better  transfer  scores  when  the 
accuracy  criterion  was  relaxed.  We  assumed  that  practice  with  a 
smaller  kill  zone  would  demand  greater  aiming  precision  and  lead 
to  a  more  refined  skill. 

We  speculated  that  practice  with  a  stricter  accuracy 
criterion  could  also  lead  to  better  retention  performance, 
occurring  in  the  following  manner:  Practice  with  a  smaller  kill 
zone  encourages  a  tighter  shot  pattern  around  the  target  center 
of  mass.  During  practice,  the  smaller  kill  zone  results  in  a 
lower  kill  rate  because  the  smaller  kill  zone  constitutes  a  more 
difficult  task.  During  the  no-practice  retention  interval, 
aiming  precision  deteriorates,  i.e.  the  shot  pattern  shows 
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greater  dispersion,  at  an  equal  rate  for  both  groups.  During 
retention  testing,  for  both  groups  more  shots  fall  outside  of  the 
boundaries  of  the  kill  zones  with  which  they  practiced.  For  the 
control  group  shots  falling  outside  of  the  practice  kill  zone  are 
misses.  However,  for  the  difficult  practice  group  some  of  the 
shots  falling  outside  of  the  original  kill  zone  setting  are  still 
within  the  (relatively)  easier  criterion  used  for  the  retention 
test.  Therefore,  on  a  retention  test  subjects  initially  trained 
with  the  stricter  criterion  would  have  a  greater  percentage  of 
their  shots  hitting  within  the  kill  zone. 

The  second  hypothesis  was  that  practice  with  an  easier 
criterion,  i.e.  the  kill  zone  set  to  150%  of  the  size  of  the 
target,  would  lead  to  better  transfer  and  retention.  An  argument 
for  this  prediction  is  that  during  practice  with  an  easy 
criterion  subjects  establish  high  performance  standards,  i.e.  a 
high  kill  rate,  which  they  endeavor  to  maintain  during  transfer 
and  retention  testing.  (This  experiment  did  not  evaluate  the 
situation  for  which  training  with  an  easier  criterion  seems  most 
appropriate:  the  need  to  train  for  a  transfer  task  that  is  too 
difficult  to  allow  for  meaningful  practice  by  naive  subjects.) 

The  first  hypothesis,  that  practice  with  a  more  difficult 
criterion  may  lead  to  better  transfer  and  retention  than  practice 
with  the  transfer  criterion,  was  our  primary  interest  because  we 
felt  it  represented  a  viable  training  strategy.  In  contrast,  the 
easy  practice  condition  employed  in  this  experiment  might  be 
expected  to  lead  to  negative  transfer  because  rounds  were  scored 
as  kills  even  though  they  hit  outside  of  the  target  silhouette. 
Although  the  easy  practice  condition  was  questionable  as  a 
practical  training  strategy  we  believed  that  the  condition  could 
help  us  interpret  the  effect  of  variations  in  practice  difficulty 
and  therefore  we  included  the  condition  in  the  experiment. 

The  experimental  design  involved  within-device  transfer. 
Positive  results  would  lead  to  consideration  of  further  research 
investigating  transfer  to  higher  fidelity  trainers,  such  as  COFT, 
or  to  live  fire  performance.  Because  this  was  exploratory 
research  we  employed  a  relatively  small  sample  size  of  15 
subjects  per  group. 


Method 


Subjects 

Forty- five  male  undergraduates  served  as  subjects.  (Fourteen 
subjects  returned  for  the  retention  test) .  Subjects  were  paid 
for  their  participation.  All  subjects  reported  normal  color 
vision. 
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Materials 


VIGS:  (A  detailed  description  of  VIGS  is  presented  in 

Appendix  A.)  The  Ml  Videodisc  Gunnery  Simulator  (VIGS)  is  a 
tabletop  part-task  trainer  for  Ml  tank  gunnery.  The  VIGS  trainer 
utilizes  computer  generated  imagery  (CGI)  to  present  engagement 
scenarios  to  the  trainee.  The  scenarios  depict  stationary  and 
moving  targets  and  realistic  terrain.  VIGS  represents  the 
controls  for  target  tracking  and  engagement,  and  the  switches  for 
ammunition  and  sight  selection,  needed  to  practice  tank  gunnery. 

Four  different  disks  (videodisks)  of  scenarios  are  available 
for  the  Ml  VIGS.  Each  VIGS  scenario  depicts  a  scene  as  viewed 
through  the  gunner's  sight  of  an  Ml  tank.  Each  scene  contains 
terrain,  such  as  hills  or  flat  stretches  of  sand.  Most  scenes 
also  contain  vegetation,  such  as  grass  and  trees,  and  cultural 
objects,  such  as  houses.  CGI  targets  are  superimposed  upon  the 
terrain  representation.  Target  types  are:  tanks,  helicopters, 
BMPs  (Warsaw  Pact  infantry  fighting  vehicles) ,  trucks,  and 
troops.  Some  scenarios  present  single  targets,  some  present  two 
targets.  Stationary  and  moving  targets  are  presented  and  some 
scenarios  represent  movement  by  the  firing  vehicle. 

From  three  of  the  VIGS  disks  we  selected  a  subset  of  the 
total  set  of  available  scenarios  (missions) .  Scenarios  that 
required  degraded  mode  gunnery  or  use  of  the  COAX  machine  gun 
were  excluded  to  avoid  the  additional  train-up  time  that  would 
have  been  required  to  prepare  subjects  for  these  kinds  of 
scenarios.  Also,  multiple  target  scenarios  were  excluded  to 
simplify  the  data  analysis,  that  is,  each  scenario  presented  only 
one  target. 


Design 

The  VIGS  capability  to  define  the  target  kill  zone  as  a 
percentage  of  the  target  silhouette  was  used  to  vary  the 
criterion  for  accuracy  required  to  score  a  kill.  A  kill  zone 
designated  as  "100%"  defined  a  rectangle  just  big  enough  to 
completely  enclose  the  target  silhouette.  The  internal 
simulation  representation  of  a  target  was  therefore  not  identical 
in  size  or  shape  of  the  visual  representation  of  the  target 
presented  to  the  trainee.  It  should  be  noted  that  the  kill  zone 
was  not  visually  observable  by  the  subject.  The  visual 
characteristics  of  the  targets  were  not  altered  by  changing  the 
kill  zone.  No  representation  of  the  kill  zone  was  superimposed 
on  the  targets.  The  apparent  size  of  each  target  was  a  direct 
function  of  the  range  to  the  target  and  the  target  type  (tank, 
helicopter,  BMP  or  truck)  and,  therefore,  target  size,  and  kill 
zone  size,  varied  across  exercises.  That  is,  the  absolu+-e  size 
of  the  kill  zone  of  a  target  was  a  direct  function  of  the  target 
size  and  the  kill  zone  setting.  The  speed  at  which  the  targets 
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moved  was  unaffected  by  the  kill  zone  setting.  The  size,  range, 
and  movement  speed  of  targets  were  constant  across  groups.  The 
differences  in  the  treatment  of  groups  were:  (1)  the  initial 
instructions  given  to  the  subject  in  regard  to  the  size  of  the 
defined  kill  zone,  and  (2)  the  kill  zone  parameter  used  by  the 
simulation  to  determine  if  a  shot  was  scored  as  a  kill. 

Subjects  were  randomly  assigned  to  groups.  Each  group 
received  three  blocks  of  practice  engagements  (trials)  with 
eighteen  engagements  per  block.  One  group  (labelled  KZ150%) 
practiced  with  the  target  kill  zone  set  at  150%  of  the  target 
silhouette;  a  second  group  (KZ100%  or  CONTROL)  practiced  with  the 
target  kill  zone  set  at  100%  of  the  target  silhouette;  a  third 
group  (KZ50%)  practiced  with  the  target  kill  zone  set  at  50%  of 
the  target  silhouette.  For  the  three  practice  blocks,  each 
scenario  was  presented  twice  in  each  block  in  random  order  except 
for  the  restriction  that  the  same  scenario  could  not  appear 
consecutively.  Five  sets  of  random  orders  of  scenarios  were  used 
for  the  blocks  of  practice  engagements.  (Complete  randomization 
of  scenario  order  across  subjects  was  precluded  by  the  difficulty 
of  the  VTGS'  procedure  for  selecting  and  storing  the  order  of 
scenario  presentation.) 

Following  practice,  all  groups  were  tested  with  a  100%  kill 
zone  using  twenty  engagements  from  either  disk  #2  or  disk  #3. 

The  targets  engaged  during  the  test  were  similar  to  those 
encountered  during  practice,  but  were  not  exact  duplicates. 
Subjects  returned  ten  weeks  after  the  test  for  a  retention  test, 
which  required  subjects  to  engage  the  set  of  18  engagements  from 
disk  #1  and  then  20  test  targets  from  the  same  disk  as  they  had 
used  during  the  original  test  session.  (Note  that  both  blocks  of 
retention  trials  were  administered  on  the  same  day.)  A  ten  week 
retention  interval  was  selected  to  allow  completion  of  the 
initial  data  collection  phase,  that  is  practice  and  testing,  and 
to  coincide  with  the  summer  semester. 

Five  random  sequences  were  created  of  the  nine  scenarios  from 
disk  #1,  with  each  scenario  appearing  twice  per  block.  Thus, 
there  were  18  engagements  in  each  of  the  three  practice  blocks. 
Each  subject  was  trained  using  3  of  the  sequences,  chosen  at 
random.  For  the  transfer  test  subjects  received  a  subset  of 
either  VIGS  disk  #2  or  #3,  in  alternating  order:  Odd  numbered 
subjects  were  tested  with  exercises  from  disk  #2,  even  numbered 
subjects  were  tested  with  exercises  from  disk  #3.  For  the 
retention  test,  subjects  received  a  block  of  exercises  selected 
from  VIGS  disk  #1  and  then,  after  a  break,  a  block  of  20 
exercises  from  the  same  VIGS  disk  (2  or  3)  that  they  had  received 
during  the  transfer  test. 

Our  initial  design  had  specified  that  subjects  would  practice 
with  scenarios  from  disk  #1,  be  tested  with  scenarios  from  a 
second  disk  and  then  receive  scenarios  from  a  third  disk  for  the 
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retention  test.  Subjects  tested  with  disk  #2  would  receive  disk 
#3  scenarios  for  the  retention  test.  Subjects  tested  with  disk 
#3  would  receive  disk  #2  for  the  retention  test.  The  use  of 
different  sets  of  VIGS  scenarios  across  the  practice,  transfer 
test,  and  retention  test  would  reduce  the  possibility  that 
subjects  were  memorizing  specific  aspects  of  individual 
scenarios. 

Unfortunately,  during  data  collection  it  became  evident  that 
kill  rates  for  several  of  the  scenarios  from  disks  #2  and  #3  were 
much  lower  than  expected.  Further  investigation  revealed 
potential  problems  with  the  VIGS  scoring  system  for  some 
scenarios  on  disks  #2  and  #3.  The  low  kill  rates  seemed  to  be 
due  to  inconsistencies  in  the  way  that  the  VIGS  system  determined 
if  a  shot  resulted  in  a  kill  or  miss.  (These  inconsistencies  are 
described  in  Appendix  B. )  Therefore,  we  modified  the 
experimental  design  by  adding  a  block  of  scenarios  from  disk  #1 
to  the  retention  test  in  addition  to  those  scenarios  from  either 
disk  #2  or  #3  which  were  used  as  a  second  block  during  retention 
testing. 

A  summary  of  the  use  of  the  VIGS  disks  is  presented  in  Table  1. 
Table  1 

VIGS  Disks  Used  for  Each  Trial  Block 

BLOCK 


Practice 


Transfer 


Retention 


12  3 


4  5  6 


VIGS  DISK  1  1  1  2  or  3  1  2  or  3 

(all  groups) 
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Procedure 


The  timeline  for  the  procedure  for  each  subject  was: 


Time 

00  -  15  minutes 
15-20  minutes 
20  -  40  minutes 
40  -  45  minutes 
45-65  minutes 
65-70  minutes 
70-90  minutes 
90  -  95  minutes 
95  -  115  minutes 
115  -  120  minutes 


Activity 

orientation 
practice 
session  I 
break 
session  II 
break 

session  III 
break 
session  IV 
debriefing/payment 


Subjects  participated  one  at  a  time.  Each  subject  was 
administered  a  human  subjects  consent  form  and  volunteer 
agreement,  (Appendix  C) .  Included  as  part  of  the  informed 
consent  form  was  an  overview  of  the  purpose  and  procedure  of  the 
research.  Next,  the  subject  completed  a  background  guest ionna ire 
(Appendix  D) . 

The  experimenter  then  read  instructions  to  the  subject. 
(Appendix  E  presents  a  complete  list  of  instructions.)  The 
instructions  described  the  experimental  task  and  included 
descriptions  of  the  VIGS  controls,  the  target  scenarios,  and  the 
target  kill  zone  setting.  The  task  reguired  the  subject  to 
manipulate  switches  and  manual  controls  of  a  tank  gunnery 
simulator  in  order  to  fire  at  target  vehicles.  Successful 
performance  of  the  task  required  several  steps.  First,  the 
subject  flipped  a  switch  to  index  the  ammunition  type  specified 
for  the  scenario.  Next,  the  subject  used  manual  controls  to 
place  aiming  crosshairs  within  the  target  silhouette.  Once  the 
crosshairs  were  on  the  target  the  subject  pressed  a  button  to 
•*lase'*  the  target,  that  is,  to  determine  the  range  to  the  target. 
The  subject  continued  tracking  the  target  for  at  least  1.5 
seconds  then  pressed  another  button  to  fire  at  the  target.  Each 
subject  fired  one  round  per  engagement.  The  subject  was 
instructed  to  observe  the  impact  of  each  round  and  then  look  away 
from  the  VIGS  sight  until  the  beginning  of  the  next  scenario. 
Following  the  instruction  period,  the  first  block  of  practice 
trials  began. 

A  five-minute  break  was  provided  after  each  block.  At  the 
beginning  of  the  fourth  block  (test)  the  experimenter  described 
the  change,  if  any,  in  the  size  of  the  kill  zone.  After  testing, 
subjects  were  paid  for  their  participation  and  informed  of  the 
planned  retention  test. 
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Subjects  returned  after  10  weeks  for  the  retention  test. 
Although  most  of  the  45  subjects  had  indicated  that  they  would  be 
willing  to  return,  only  14  of  the  subjects  could  be  contacted  and 
scheduled;  four  from  KZ100%  and  five  each  from  KZ50%  and  KZ150%. 
(The  high  attrition  rate  may  be  attributable  to  the  scheduling  of 
the  retention  test  during  the  summer  semester.) 

For  the  retention  test  each  subject  was  tested  with  the 
original  set  of  disk  #1  scenarios.  After  a  five-minute  rest  each 
subject  was  tested  with  either  the  disk  #2  or  disk  #3  scenarios. 

Engagements  during  which  subjects  made  a  procedural  error, 
such  as  indexing  the  wrong  ammunition,  were  excluded  from  the 
analyses.  Procedural  errors  prevent  the  determination  of  the 
subject's  aiming  point  at  the  time  of  firing  because  the  point  of 
round  impact  does  not  correspond  to  the  aiming  point. 


Results 

Table  2  presents  group  means  and  standard  deviations  for  the 
three  dependent  variables,  time  to  fire,  kill  percentage,  and 
aiming  error,  for  each  block  of  trials.  Group  means  across  trial 
blocks  are  plotted  for  kill  percentage,  time  to  fire,  and  aiming 
error  in  Figures  l,  2  and  3  respectively.  In  the  tables  and 
figures,  the  first  block  of  trials  of  the  retention  test  is 
labelled  Form  1,  the  second  block  is  labelled  Form  2.  These 
labels  were  chosen  to  indicate  that  the  two  blocks  of  retention 
trials  used  different  sets  of  scenarios  but  do  not  represent 
different  retention  intervals. 
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Table  2 


Means/Standard  Deviations  for  Time  to  Fire  (TF) ,  Kill  Percentage 
(K%) ,  and  Aiming  Error  (AE)  per  Group  by  Trial  Block 


Measure 

Block  Group 

KZ% 

TF 

M 

(SD) 

M 

K% 

(SD) 

AE 

M 

(SD) 

1  50 

11.34 

(3.03) 

45 

(18) 

1.72 

(0.33) 

100 

9.84 

(2.52) 

58 

(19) 

2.58 

(1.25) 

150 

9.27 

(1.57) 

65 

(17) 

2.93 

(1.97) 

2  50 

10.56 

(2.57) 

59 

(12) 

1.35 

(0.45) 

100 

8.67 

(1.96) 

72 

(19) 

1.75 

(0.53) 

150 

8.07 

(1.16) 

80 

(10) 

1.85 

(0.74) 

3  50 

10.40 

(2.42) 

59 

(12) 

1.31 

(0.26) 

100 

8.09 

(1.47) 

76 

(17) 

1.52 

(0.43) 

150 

7.82 

(1.26) 

85 

(10) 

1.46 

(0.62) 

4  (Test)  50 

11.57 

(2.55) 

44 

(13) 

2.01 

(0.41) 

100 

10.16 

(2.70) 

36 

(13) 

2.33 

(0.53) 

150 

9.87 

(2.69) 

36 

(14) 

2.69 

(1.10) 

Retention  50 

9.28 

(2.07) 

85 

(07) 

1.26 

(0.31) 

(Form  1)  100 

8.39 

(0.78) 

77 

(11) 

1.60 

(0.78) 

150 

8.40 

(1.59) 

80 

(09) 

1.47 

(0.29) 

Retention  50 

11.32 

(3.81) 

36 

(05) 

2.18 

(0.66) 

(Form  2)  100 

9.26 

(2.05) 

30 

(12) 

2.38 

(0.30) 

150 

9.04 

(2.48) 

31 

(15) 

2.71 

(1.26) 

1  (N)  1 

Differ  50(4) 

.  15 

(1.99) 

18 

(15) 

-.40 

(0.97) 

-ence  100(4) 

.54 

(1.48) 

6 

(08) 

.01 

(0.29) 

Score*  150(5) 

1.08 

(1.52) 

6 

(19) 

-.10 

(0.77) 

“  Difference  score  =  Test  score  -  Second  Retention  Block  Score 
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Practice 


The  group  means  for  kill  percentage  are  shown  in  Figure  1. 
For  each  of  the  three  blocks  of  practice  trials  KZ150%  had  the 
highest  kill  percentage  followed  by  KZ100%  and  then  KZ50%.  This 
pattern,  as  was  expected,  corresponds  to  the  differential 
criteria,  determined  by  kill  zone  setting,  employed  during  the 
practice  blocks. 


Practice 


Porm  1  Form  2 


Retention 


TRIAL  BLOCKS 


Figure  1.  Mean  target  kills  (%)  as  functions  of  blocks  of 
practice,  transfer,  and  retention  trials  for  each  of  three  kill 
zone  percentages 
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The  group  means  for  aiming  error  are  shown  in  Figure  3. 
(Aiming  error  is  a  measure  of  absolute  distance  (Pythagorean 
function  of  azimuth  and  elevation  deviation)  from  target  center 
of  mass  to  round  impact,  measured  in  milliradians,  a  measure  of 
visual  angle.)  KZ50%  displayed  a  consistent  superiority 
(smallest  mean  error)  across  all  blocks. 
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Test  Retention 


TRIAL  BLOCKS 


Figure  3.  Mean  aiming  error  as  functions  of  blocks  of  practice, 
transfer,  and  retention  trials  for  each  of  three  kill  zone 
percentages 
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A  doubly  multivariate  analysis  of  variance  (MANOVA)  was 
conducted  on  practice  performance  (trial  blocks  1-3) ,  with  a 
between-subjects  factor  of  group  assignment,  a  within-subjects 
factor  of  trial  block,  and  three  dependent  variables  (kill 
percentage,  time  to  fire,  and  aiming  error).  Wilks'  lambda  was 
used  to  determine  statistical  significance.  (All  analyses  were 
conducted  using  the  Statistical  Package  for  the  Social  Sciences.) 

MANOVA  results  are  reported  in  Table  3  for  the  doubly 
multivariate  analysis,  followed  by  each  singly  multivariate 
analysis.  For  the  practice  data,  significant  main  effects  were 
found  for  group  assignment  and  trial  block,  indicating  that 
gunnery  performance  differed  as  a  function  of  group  assignment 
and  as  a  function  of  amount  of  practice.  The  interaction  of 
group  assignment  and  block  was  not  significant. 


Table  3 

MANOVA  Summary  Tables 


EFFECT 

WILKS ' 

APP.  F 

HYP.  DF 

ERR.  DF 

SIG. 

PRACTICE  BLOCKS; 

GRP 

.36 

8.93 

6 

80 

.000 

BLOCK 

.27 

16.86 

6 

37 

.000 

GRP  X  BLOCK 

.68 

1.32 

12 

74 

.228 

TEST  BLOCK: 

GRP 

.75 

2.11 

6 

80 

.061 

RETENTION  BLOCK 

1: 

GRP 

.80 

.36 

6 

18 

.895 

RETENTION  BLOCK 

2: 

GRP 

.73 

.45 

6 

16 

.838 

Note.  Dependent  measures  =  kill  percentage,  time  to  fire,  and 
aiming  error. 
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To  further  investigate  the  significant  main  effects  found  for 
practice  performance,  univariate  ANOVAs  were  run  for  each  of  the 
dependent  variables  (See  Tables  4, 5, and  6).  The  results  of  each 
of  these  analyses  paralleled  the  MANOVA  results:  group  and  trial 
effects  were  significant  with  no  significant  interactions. 


Table  4 

ANOVA  Summary  Table  for  Kill  Percentage  by  Practice  Blocks 


SOURCE 

DF 

F 

p  LEVEL 

MAIN  EFFECTS: 

GRP 

2,42 

12.91 

.000 

BLOCK 

2,84 

29.66 

.000 

INTERACTION: 

GRP  X  BLOCK 

4,84 

.28 

.  888 

Table  5 

ANOVA  Summary  Table 

for  Time  to  Fire  by 

Practice  Blocks 

SOURCE 

DF 

F 

p  LEVEL 

MAIN  EFFECTS: 

GRP 

2,42 

5.98 

.005 

BLOCK 

2,84 

37.94 

.  000 

INTERACTION: 


GRP  X  BLOCK 


4,84 


1.11 


.  356 


Table  6 


ANOVA  Summary  Table  for  Aiming  Error  by  Practice  Blocks 


SOURCE 

DF 

F 

p  LEVEL 

MAIN  EFFECTS: 

GRP 

2,42 

4.28 

.020 

BLOCK 

2,84 

18.43 

.000 

INTERACTION: 

GRP  X  BLOCK 

4,84 

1.74 

.149 

To  determine  significant  differences  among  groups,  Duncan 
multiple  range  tests  were  conducted  for  each  of  the  three 
dependent  variables.  For  kill  percentage,  time  to  fire,  and 
aiming  error  the  means  for  KZ50%  differed  significantly  (p<.05) 
from  the  means  for  KZ100%  and  KZ150%,  which  did  not  differ 
significantly.  The  range  tests  were  based  on  one-way  ANOVAs  of 
the  variables  collapsed  across  practice  trials.  All  of  the 
ANOVAs  were  significant  (g<.05). 

Transfer  and  Retention 


Separate  MANOVAs,  see  Table  3,  were  conducted  on  the  transfer 
test  (trial  block  4) ,  and  the  two  blocks  of  the  retention  test. 
Results  of  these  MANOVAs  were  not  significant,  although  results 
for  the  transfer  test  (block  4)  analysis  (F  =  2.112,  d.f.  =  6, 

80)  approached  significance  (p  =  .061).  (The  small  number  of 
subjects  participating  in  retention  testing  greatly  limited  the 
power  of  the  analyses  and  impeded  evaluation  of  the  effects  of 
the  experimental  treatments  on  retention.) 

Separate  ANOVAs  were  conducted  for  each  of  the  dependent 
variables  for  the  test  block  (Table  7),  retention  block  1  (Table 
8)  and  retention  block  2  (Table  9) .  None  of  these  analyses  were 
significant,  although  for  aiming  error  the  main  effect  of  group 
(F  =  3.07,  df  =  2,42)  during  the  test  block  (see  Table  7) 
approached  significance  with  a  p  level  of  .057. 
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Table  7 


ANOVAs  Summary  Table  Across  Groups  for  Kill  Percentage,  Time  to 
Fire,  and  Aiming  Error  by  Test  Block 


DEP.  VAR. 

DF 

F 

p  LEVEL 

KILL  PERCENTAGE 

2,42 

1.85 

.  170 

TIME  TO  FIRE 

2,42 

1.77 

.  182 

AIMING  ERROR 

2,42 

3.07 

.057 

Table  8 

ANOVAs  Summary  Table  Across 
Fire,  and  Aiming  Error  for 

Groups  for  Kill 
Retention  Form  1 

Percentage 

,  Time  to 

DEP.  VAR. 

DF 

F 

p  LEVEL 

KILL  PERCENTAGE 

2,11 

.94 

.419 

TIME  TO  FIRE 

2,11 

.47 

.635 

AIMING  Error 

2,11 

.57 

.581 

Table  9 

ANOVAs  Summary  Table  Across 
Fire,  and  Aiming  Error  for 

Groups  for  Kill 
Retention  Form  2 

Percentage 

,  Time  to 

DEP.  VAR. 

DF 

F 

p  LEVEL 

KILL  PERCENTAGE 

2,10 

.27 

.769 

TIME  TO  FIRE 

2,10 

.83 

.466 

AIMING  ERROR 

2,10 

.40 

.679 

1 


Group  means  for  kill  percentage  are  shown  in  Figure  1.  For 
the  blocks  of  trials  for  which  all  groups  encounter  the  same  kill 
zone  setting,  that  is,  the  transfer  test  and  the  two  blocks  of 
retention  testing,  the  mean  kill  percentage  for  KZ50%  was 
consistently  higher  than  the  means  for  KZ100%  or  KZ150%.  Group 
means  for  time  to  fire  are  shown  in  Figure  2.  KZ50%  had  the 
highest  (slowest)  time  to  fire  across  blocks.  KZ100%  and  KZ150% 
had  similar  means  across  all  blocks.  Group  means  for  aiming 
error  are  shown  in  Figure  3.  KZ50%  consistently  had  the  lowest 

aiming  error  across  all  blocks. 

Table  2  presents  mean  difference  scores  derived  by 
subtracting  each  subject's  score  on  the  second  block  of  the 
retention  test  from  the  subject's  score  on  the  transfer  test. 
KZ50%  showed  the  greatest  change  in  accuracy  with  the  largest 
losses  in  both  kill  percentage  and  aiming  accuracy.  All  groups 
showed  a  slight  increase  in  firing  speed. 

Difference  scores  were  also  calculated  for  KZ100%,  the 
control  group,  by  subtracting  the  retention  block  1  scores  from 
the  last  practice  block  scores.  Kill  rate  declined  by  2 
percentage  points,  time  to  fire  was  .5  seconds  slower,  and  aiming 
error  increased  by  .17  milliradians. 

A  Cronbach's  alpha  measure  of  test  reliability  was  calculated 
for  each  of  the  tests,  comprised  of  disk  2  scenarios  and  disk  3 
scenarios,  used  for  the  transfer  test.  For  the  test  made  up  of 
disk  2  scenarios  alpha  was  .53.  For  the  test  made  up  of  disk  3 
scenarios  alpha  was  .56. 


Discussion 


Practice 


During  acquisition  (practice)  trials  the  kill  zone  setting 
manipulation  resulted  in  significant  performance  differences 
among  groups.  The  observed  differences  in  kill  rate  resulted  as 
a  direct  function  of  the  kill  zone  size,  that  is,  if  the  kill 
zone  setting  manipulation  had  produced  no  change  in  subject 
behavior  we  would  still  observe  differences  across  groups  simply 
as  a  function  of  differences  in  kill  zone  size.  However, 
significant  differences  were  also  found  for  the  other  dependent 
variables:  time  to  fire  and  aiming  error.  The  groups  did  perform 
differently  even  though  the  manipulation  of  kill  zone  setting 
produced  no  visible  change  in  the  target  presentation.  The  group 
under  the  reduced  kill  zone  condition  had  slower  firing  times  and 
less  aiming  error  than  the  control  group  and  the  expanded  kill 
zone  group.  The  requirement  for  all  subjects  to  lase  within  the 
target  silhouette,  regardless  of  kill  zone  size,  probably  tended 
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to  mitigate  any  effect  of  kill  zone  size  for  the  expanded  kill 
zone  group. 

The  finding  that  a  speed/ accuracy  trade-off  occurred  during 
the  practice  trials  is  consistent  with  previous  research;  Fitts 
and  Posner  (1967)  emphasized  that  the  performance  of  many  skills 
displays  a  speed/accuracy  trade-off.  However,  the  direction  of 
the  trade-off,  accuracy  emphasized  over  speed  for  the  reduced 
kill  zone  setting,  differs  from  results  reported  by  Abel  (1986) 
which  indicated  an  emphasis  on  speed  over  accuracy  for  a  similar 
task  with  a  reduced  kill  zone  setting.  Our  procedure  limited 
subjects  to  one  round  per  engagement  which  may  have  resulted  in 
an  emphasis  on  firing  accuracy  rather  than  speed  of  engagement. 

Rabbit  (1989)  noted  that  for  research  involving  self-paced 
tasks  the  experimental  instructions  usually  direct  the  subjects 
to  be  as  fast  and  as  accurate  as  possible.  Rabbit  pointed  out 
that  these  instructions  leave  the  subjects  to  decide  what  is  the 
appropriate  emphasis  for  speed  versus  accuracy.  For  subsequent 
research  an  attempt  should  be  made  to  restrict  variation  in 
subject  performance  to  either  the  speed  or  accuracy  dimension. 

For  example,  a  relatively  brief  response  time  limit  could  be 
imposed  to  reduce  response  time  variation. 

Transfer  and  Retention 

For  practical  training  applications  the  most  useful  finding 
would  have  been  that  for  the  transfer  test,  or  the  retention 
test,  either  or  both  of  the  experimental  conditions  was  superior 
to  the  control  condition.  Superior  performance  would  be  defined 
as  some  improvement  in  both  speed  and  accuracy,  or,  an 
improvement  in  one  with  no  offsetting  decrement  in  the  other. 

This  was  not  found.  The  group  that  practiced  with  the  reduced 
kill  zone  displayed  greater  accuracy,  in  both  kill  rate  and 
aiming  error,  relative  to  the  control  group  (the  difference  was 
statistically  nonsignificant) ,  but  suffered  slower  firing  times. 

Although  the  experimental  hypotheses  were  not  supported,  the 
results  of  this  experiment  have  implications  for  the  conduct 
of  subsequent  research.  These  implications  are: 

1.  The  manner  in  which  we  attempted  to  use  the  VIGS  tank 
gunnery  training  device  as  a  research  device  was  not  appropriate. 
Some  of  the  characteristics  that  probably  increase  the 
effectiveness  of  VIGS  as  a  training  device  decrease  the 
effectiveness  of  VIGS  as  a  research  device.  For  example,  the 
VIGS  scenarios  vary  greatly  in  difficulty.  Some  of  the  scenarios 
provide  relatively  easy  engagement  of  slow  targets  moving  at  a 
constant  rate.  For  other  scenarios,  however,  the  targets 
unpredictably  change  speed  and  direction.  Although  these 
scenarios  provide  challenging  engagements  for  even  experienced 
gunners,  as  research  materials  the  scenarios  do  not  allow  the 
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identification,  measurement,  or  control  of  the  dimensions  of 
scenario  difficulty. 

2.  Unexpectedly  low  kill  rates  were  obtained  with  some  of 
the  VIGS  scenarios.  The  collection  and  graphic  display  of 
azimuth  and  elevation  aiming  error  allowed  for  a  more  meaningful 
interpretation  of  performance  than  could  have  been  obtained  from 
only  kill  rate  measurement.  In  addition,  analysis  of  the 
direction  of  error — left  versus  right  for  azimuth,  high  versus  low 
for  elevation — seems  preferable  to  sacrificing  this  information 

by  computing  the  geometric  aim  deviation  without  regard  to  the 
direction  of  deviation.  Analysis  of  shot  patterns  for  individual 
scenarios  may  reveal  systematic  deviation,  in  azimuth  or 
elevation,  from  target  center  of  mass.  This  deviation  may  be 
indicative  of  problems  with  the  training  scenario  or  systematic 
problems  in  trainee  performance. 

3.  An  experimental  procedure  that  limits  subjects  to  one 
round  per  engagement  places  a  greater  emphasis  on  accuracy  rather 
than  speed  of  engagement.  Limiting  subjects  to  one  round  per 
engagement  avoids  cross  subject  variation  in  the  number  of  rounds 
fired  during  acquisition  and  testing.  However,  this  limitation 
imposes  task  demand  characteristics  that  differ  from  standard  use 
of  gunnery  simulators  and  live  fire  conditions  for  which  multiple 
firing  is  allowed  and  sometimes  required. 

Finally,  this  research  has  implications  for  tank  gunnery 
training.  Training  approaches  that  encourage  or  require  trainees 
to  engage  targets  with  a  small  kill  zone  may  tend  to  produce 
slower  firing  times.  Training  managers  should  weigh  the  benefits 
of  increased  accuracy  against  the  loss  in  speed  of  firing. 
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APPENDIX  A 


DESCRIPTION  OF  VIGS 

VIGS:  The  Ml  Videodisc  Gunnery  Simulator  (VIGS)  is  a 

tabletop  part-task  trainer  for  Ml  or  MlAl  tank  gunnery.  The  VIGS 
trainer  utilizes  computer  generated  imagery  (CGI)  to  present 
engagement  scenarios  to  the  trainee.  The  scenarios  depict 
stationary  and  moving  targets  and  realistic  terrain.  The  CGI 
scenarios,  and  a  series  of  target  identification  slides,  are 
stored  on  videodisk.  The  scenarios  are  of  approximately  45  to  60 
seconds  duration.  Through  the  use  of  synthesized  speech,  a 
simulated  "tank  commander"  issues  standard  fire  commands, 
specifying  the  target  type,  required  ammunition,  and  fire 
directives.  At  the  end  of  the  engagement,  trainees  can  be 
provided  detailed  performance  measures  via  a  screen  display  and 
by  an  optional  on-line  printer. 

The  VIGS  performance  measures  are  presented  on  an  embedded 
cathode  ray  tube,  and  are  separated  into  3  screens.  The  first 
screen  is  titled  the  "Scoring  Screen",  and  includes  the  following 
information:  Trainee  Point  Score,  calculated  according  to  a 

formula  which  combines  elements  of  speed,  accuracy  and  procedural 
performance;  Possible  Point  Score,  the  total  point  score 
attainable;  the  length  of  the  scenario  in  seconds;  the  Total 
Session  Score,  reflecting  a  cumulative  record  of  trainee 
performance;  Session  Performance,  a  cumulative  percentage  score 
based  on  session  score;  Rating,  which  may  be  Distinguished, 
Superior,  Qualified,  or  Unqualified,  depending  upon  the  trainee's 
Session  Performance  score;  Total  Rounds  Fired  and  Remaining  for 
both  main  gun  and  COAX  ammunition;  COAX  Target  Coverage;  and 
detailed  information  regarding  each  round  fired.  This 
information  includes  point-of-impact  azimuth  and  elevation 
displacement  in  milliradians  from  target  center  of  mass. 

The  second  screen  is  entitled  the  "Critique  Screen"  and 
includes  verbal  descriptive  phrases  informing  trainees  of 
specific  inadequate  performance.  Time  taken  to  kill  the  target 
is  also  included  on  this  screen. 

The  third  screen  is  the  Summary  Screen,  and  repeats  the 
azimuth  and  elevation  displacement  of  each  round  given  on  the 
Scoring  Screen.  Also,  time  taken  to  fire  the  round  is  shown,  as 
is  the  round's  effectiveness  (hit  or  miss).  Because  of  the 
complexity  of  the  information  displayed  on  the  VIGS  screens  a 
shield  was  placed  at  the  side  of  the  cathode  ray  tube  so  that  the 
subject  could  not  view,  and  be  distracted  by,  the  display  during 
the  experiment.  The  shield  did  not  block  the  view  of  the 
experimenter . 
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For  this  research,  the  VIGS  was  modified  to  include  a 
printer,  used  to  create  hard  copies  of  performance  reports 
(information  dumps  from  the  VIGS  data  screens)  and  a  keyboard, 
used  to  initiate  the  printouts  and  to  alter  some  of  VIGS' 
parameters. 

Four  different  videodisks  of  scenarios  are  available  for  the 
Ml  VIGS.  Each  VIGS  scenario  depicts  a  scene  as  viewed  through 
the  gunner's  sight  of  an  Ml  tank.  Each  scene  contains  terrain, 
such  as  hills  or  flat  stretches  of  sand.  Most  scenes  also 
contain  vegetation,  such  as  grass  and  trees,  and  cultural 
objects,  such  as  houses.  CGI  targets  are  superimposed  upon  the 
terrain  representation.  Target  types  are;  tanks,  helicopters, 
BMPs  (Warsaw  Pact  infantry  fighting  vehicles) ,  trucks,  and 
troops.  Some  scenarios  present  single  targets,  some  present  two 
targets.  Stationary  and  moving  targets  are  presented  and  some 
scenarios  represent  movement  by  the  firing  vehicle. 
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APPENDIX  B 


DISCUSSION  OF  PROBLEMS  WITH  VIGS  SCENARIOS 

Target  kill  rates  for  some  of  the  scenarios  from  VIGS  disks  2 
and  3  were  much  lower  than  target  kill  rates  for  scenarios  from 
disk  1.  This  appendix  describes  our  efforts  to  determine  the 
basis  of  the  lower  kill  rates,  the  identification  of  possible 
technical  problems  with  some  of  the  scenarios,  and  the  training 
implications  of  the  problems. 

Many  of  the  subjects  complained  during  the  transfer  test  that 
on  several  of  the  test  scenarios  their  shots  were  incorrectly 
being  scored  as  misses.  These  subjects,  who  had  developed 
confidence  during  the  three  blocks  of  practice  trials,  claimed 
that  they  had  correctly  engaged  targets  but  the  VIGS  system  had 
incorrectly  computed  misses  instead  of  kills.  For  the  two  worst 
cases,  mission  22  from  disk  2  and  mission  6  from  disk  3,  less 
than  5  percent  of  the  rounds  fired  were  recorded  as  kills. 

For  each  of  the  VIGS  exercises  used  in  this  experiment 
graphs  of  subjects'  shot  patterns  were  generated.  The  graphs 
depict  the  dispersion,  in  milliradians,  of  rounds  with  the  origin 
of  the  graph  corresponding  to  the  center  of  the  target.  (The 
numbers  and  letters  indicate  the  number  of  rounds  falling  at  a 
single  plot  point.  Because  two  digit  numbers  can  not  be  printed 
at  a  single  point  letters  are  used:  A=10,  B=  11,  ...,  Z  =  35,  *  = 
36  or  more) .  Many  of  these  patterns  show  rounds  centered  around 
the  graph  origin,  as  we  expected.  These  patterns  indicate  that 
rounds  consistently  hit  near  the  target  center  with  the  dispersal 
pattern  centered  on  the  origin.  An  example  of  this  type  of 
result  is  the  shot  pattern  for  scenario  5  (Mission  5)  from  disk 
1.  This  graph  (Figure  4)  shows  shots  evenly  clustered  about  the 
origin. 
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Most  of  the  shot  patterns  for  disk  1  scenarios  were  evenly 
distributed  about  the  center  of  the  target.  In  contrast,  many  of 
the  patterns  for  scenarios  from  disks  2  and  3  were  not  centered 
around  the  graph  origin;  the  majority  of  the  rounds  fell  above 
the  target  center.  An  example  of  these  unexpected  shot  patterns 
is  shown  in  Figure  5. 


MISSION  23  DISK  2 


I  I  I  I  I  I 


-4.5  -1.5  1.5  4.5 

-3  0  3 


AZIMUTH 

Figure  5.  Shot  pattern  for  VIGS  Mission  23  from  Disk  2 


The  instructions  to  the  subjects  described  only  the  basic 
procedures  required  to  track  and  engage  targets.  The 
instructions  did  not  cover  complex  aspects  of  tank  gunnery,  such 
as  multiple  returns  from  the  laser  range  finder,  and  tracking 
procedures  for  engagements  in  which  target  movement  is  not 
limited  to  a  horizontal  plane.  Successful  engagement  of  some  of 
the  targets  from  disks  2  and  3  may  require  che  use  of  gunnery 
procedures  beyond  the  scope  of  the  instructions  and  practice 
provided  to  subjects  in  this  experiment.  However,  the  low  kill 
rates  recorded  for  some  of  the  scenarios  from  disks  2  and  3  may 
be  the  result  of  technical  problems  with  those  scenarios. 
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Informal  investigation  of  the  "problem"  scenarios  indicated 
that  there  may  be  a  discrepancy  between  the  screen  representation 
of  the  location  of  the  target,  where  the  target  appears  to  be, 
and  the  system  representation,  the  coding  of  target  location  used 
to  determine  if  a  shot  is  a  hit  or  miss.  Discrepancies  were 
determined  by  repeatedly  lasing  within  and  slightly  outside  of 
the  target  silhouettes.  For  example,  for  several  of  the 
scenarios,  lasing  on  the  upper  third  of  the  target  produces  a 
range  figure  which  is  approximately  400  to  600  meters  more 
distant  than  the  figure  which  results  from  lasing  on  the  lower 
two  thirds  of  the  target.  The  discrepancy  in  range  determination 
produces  a  corresponding  discrepancy  in  firing  accuracy.  When  a 
trainee  fires  after  lasing  on  the  upper  portion  of  the  target, 
the  round  lands  beyond  the  target;  however,  when  the  trainee 
fires  after  lasing  on  the  bottom  portion  of  the  target,  or  just 
below  the  target,  the  round  will  hit  the  target. 

Representatives  of  the  VIGS  manufacturer,  the  Education 
Computers  Corporation,  were  consulted  to  try  to  resolve  our 
questions  concerning  the  VIGS  scenarios.  The  representatives 
pointed  out  that  VIGS  was  developed  to  serve  as  a  low-cost 
training  device.  They  acknowledged  that  for  a  few  of  the 
scenarios  there  could  be  minor  discrepancies  between  the  visual 
representation  of  target  location  and  the  internal  system 
representation  of  target  location.  Discrepancies  could  also 
exist  between  the  representations  of  the  aiming  reticle.  They 
expressed  the  view  that  these  discrepancies  did  not  compromise 
the  training  effectiveness  of  VIGS. 

Experienced  tank  gunnery  instructors  could  be  used  to  test 
the  scenarios  from  disks  2  and  3  to  determine  if  technical 
problems  exist  and  to  provide  some  indication  of  the  problems' 
potential  effects  on  training  effectiveness.  An  acceptable 
percentage  of  first  round  kills  would  indicate  that  the  VIGS 
scenarios  functioned  properly  and  that  the  low  rates  we  observed 
were  the  result  of  inadequate  instructions  or  practice  given  to 
our  subjects.  However,  if  for  some  of  the  scenarios  very  low 
kill  rates  are  obtained  even  with  experienced  gunners  then  this 
information  should  be  made  available  to  the  managers  of  VIGS 
training. 

Training  managers  may  wish  to  omit  the  problem  scenarios 
during  VIGS  training.  It  would  seem  inappropriate  to  use 
scenarios  for  which  proper  engagements  techniques  sometime  lead 
to  unexplained  misses  and  improper  techniques,  such  as  aiming 
below  the  target,  result  in  kills.  On  the  other  hand,  the 
managers  may  be  able  to  incorporate  the  problem  scenarios  into 
the  training  plan,  for  example,  by  using  them  to  explicitly 
demonstrate  an  aspect  of  live  fire;  sometimes  the  gunner 
correctly  performs  all  procedures,  obtains  a  correct  sight 
picture,  and  yet  a  round  still  misses. 
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APPENDIX  C  (SIDE  1) 


HUMAN  SUBJECTS  CONSENT  FORM  AND  VOLUNTEER  AGREEMENT 


I,  _ ,  having 

full  capacity  to  consent,  do  hereby  volunteer  to  participate  in 
research  entitled  Application  of  Learning  Principles  to  Device- 
based  Training  under  supervision  of  the  US  Army  Research 
Institute.  The  implications  of  my  voluntary  participation  and 
the  nature,  duration,  and  purpose  of  the  research,  and  the  method 
and  means  by  which  it  is  to  be  conducted  are  contained  on  the 
reverse  side  of  this  form.  I  have  been  given  an  opportunity  to 
read  and  keep  a  copy  of  this  Agreement  and  to  ask  questions 
concerning  this  research.  Any  such  questions  have  been  answered 
to  my  full  and  complete  satisfaction.  Should  any  further 
questions  arise,  I  will  be  able  to  contact  Don  Lampton  at  380- 
4368.  I  understand  that  I  may  at  any  time  during  this  research 
revoke  my  consent  and  withdraw  from  the  test  without  prejudice, 
but  I  will  not  be  paid  unless  I  complete  2  full  hours. 


(Signature,  Date) 


I  was  present  during  the  explanations  referred  to  above  as 
well  as  the  volunteer's  opportunity  for  questions  and  hereby 
witness  his  signature. 


(Witness) 


(Date) 
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APPENDIX  C  (Side  2) 


HUMAN  SUBJECTS  CONSENT  FORM  AND  VOLUNTEER  AGREEMENT 

For  many  educational,  industrial  and  military  applications 
computer-based  training  simulations  provide  an  inexpensive  and 
safe  complement  to  training  with  operational  systems  and 
equipment.  This  experiment  is  part  of  a  project  to  develop 
better  methods  of  training  with  simulators.  The  purpose  of  this 
experiment  is  to  evaluate  training  effectiveness  as  a  function  of 
the  relation  between  the  level  of  difficulty  of  early  stages  of 
training  and  the  level  of  difficulty  of  later  stages  of  training. 

You  will  be  asked  to  practice  an  aiming  task  using  a 
computer-based  training  simulator  which  was  developed  to  train 
tank  gunnery.  You  will  aim  and  fire  at  a  series  of  targets. 

After  each  shot  you  will  receive  an  indication  of  whether  you  hit 
or  missed.  The  task  involves  setting  a  series  of  switches  and 
using  manual  controls  to  track  a  target.  (The  device  display  and 
the  required  task  are  somewhat  similar  to  the  displays  and  tasks 
used  for  many  video  games.)  After  an  initial  practice  period  you 
will  complete  4  sessions  of  tracking  tasks.  Each  session  will 
consist  of  approximately  18  targets.  The  degree  of  aiming 
accuracy  required  to  produce  a  hit  may  vary  across  sessions. 
Required  accuracy  will  be  explained  at  the  beginning  of  each 
session. 


Schedule: 

Time  Activity 


00  - 

15  minutes 

orientation 

15  - 

20  minutes 

practice 

20  - 

40  minutes 

session  I 

40  - 

45  minutes 

break 

45  - 

65  minutes 

session  II 

65  - 

70  minutes 

break 

70  - 

90  minutes 

session  III 

90  - 

95  minutes 

break 

95  - 

115  minutes 

session  IV 

115 

-  120  minutes 

debriefing/payment 

The  risks  involved  are  those  associated  with  viewing  standard 
video  display  screens. 

You  are  free  to  terminate  participation  in  this  experiment  at 
any  time  without  bias.  However,  you  will  not  be  paid  unless  you 
complete  the  2  hour  session. 

Do  you  have  any  questions? 
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APPENDIX  D 


SUBJECT  BACKGROUND  INFORMATION 

The  purpose  of  this  questionnaire  is  to  collect  background 
information  on  participants  in  the  ARI  tank  gunnery  training 
research.  This  information  will  be  used  strictly  for  research 
purposes  only.  Please  complete  each  item  to  the  best  of  your 
ability.  Write  "N/A"  for  each  item  you  cannot  answer. 


1. 

Name: _ 

Last  First  M.I. 

2.  Social  Security  Number: _ - _ - _ 

3.  Date  of  Birth: _ / _ / _ 

4.  Present  grade  classification  (Junior,  Senior, 

etc . ) _ 


5.  How  often  do  you  play  video  games  (circle  one)? 

A.  less  than  once  per  week 

B.  once  per  week 

C.  2 “4  times  a  week 

D.  greater  than  4  times/week 

6.  Have  you  ever  been  diagnosed  as  color  blind/deficient? 
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APPENDIX  E 


INSTRUCTIONS  TO  SUBJECTS 

Hello.  My  name  is  _  with  the  Army  Research 

Institute.  Today  you  will  train  for  approximately  2  hours  on  the 
VIGS  tank  gunnery  trainer. 

Please  have  a  seat  in  front  of  the  trainer.  Look  at  the  two 
connected  handles.  These  handles  move  the  aiming  crosshairs  up, 
down,  and  side  to  side.  To  move  the  crosshairs  side  to  side,  turn 
the  handles  like  a  steering  wheel.  To  move  them  up  or  down,  twist 
the  handles  accordingly,  (demonstrate) 

You  will  also  notice  two  sets  of  buttons.  The  first  set  of 
buttons,  located  near  the  top  and  inner  portions  of  the  handles, 
controls  the  laser  rangefinder.  This  gives  you  a  "range  lock"  on 
the  target,  and  computes  the  target's  range,  which  is  then  shown 
on  the  screen.  The  second  set  of  buttons,  located  near  your  index 
fingers,  are  the  fire  buttons.  Finally,  in  order  for  any  buttons 
or  movements  to  work,  THE  PALM  LEVERS  ON  THE  FRONT  OF  THE  HANDLES 
MUST  BE  ENGAGED! ! ! 

When  engaging  a  target,  the  sequence  of  activities  is: 

1.  Squeeze  the  palm  levers  and  hold  them  down  throughout  the 
engagement . 

2.  Manipulate  the  handles  to  bring  the  crosshairs  to  the 
center  of  the  target. 

3.  When  the  crosshairs  are  on  the  target  press  the  lase 
buttons.  For  moving  targets  you  must  track  the  target  for 
at  least  one  and  a  half  seconds  before  you  fire. 

4 .  After  the  tank  commander  gives  the  "FIRE"  command , 
press  the  fire  button,  keeping  the  crosshairs  on  the 
center  of  the  target. 

5.  Continue  tracking  the  target  until  the  round  lands. 

Look  away  from  the  sight  after  the  round  lands. 

YOU  MUST  FIRE  ONLY  ONE  ROUND  AT  EACH  TARGET  PRESENTED! 

Look  at  the  panel  in  front  of  you.  Before  an  engagement  begins,  I 
will  indicate  what  type  of  engagement  to  expect.  There  are  two 
possibilities: 

1.  "Normal  Engagement",  in  this  case,  the  shutter  switch 
needs  to  be  set  on  "clear"  and  the  thermal  mode 
switch  .  must  be  on  "standby"  (Demonstrate) . 
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2.  "Thermal  Engagement".  In  this  case,  the  shutter  switch 
must  be  turned  clockwise  to  "SHUTTR".  Also,  the 
thermal  mode  switch  must  be  set  to  "ON" 

(Demonstrate) . 


After  proper  setting  of  the  shutter  and  thermal  mode  switches, 
the  engagement  will  begin.  An  automated  tank  commander  will  move 
you  near  the  target,  while  telling  you  the  type  of  ammunition  to 
use  (i.e.,  "GUNNER,  SABOT"  or  "GUNNER,  HEAT"). 

After  hearing  this,  you  must  manipulate  the  ammunition  selection 
switch  accordingly. 

At  this  point,  you  should  place  the  crosshairs  on  the  target, 
press  the  lase  button,  and  fire,  continuously  tracking  the  target 
until  the  round  falls. 

During  the  first  practice  sessions,  the  required  accuracy  for 
target  destruction  will  be  either  smaller,  larger,  or  the  same  as 
that  for  the  last  session  (the  test  session) . 

«At  this  point,  read  appropriate  group  instructions» 

Instructions  for  subjects  in  150%  group 

For  the  first  three  blocks  of  trials  the  target  kill  zone  is  set 
so  that  a  shot  that  lands  slightly  outside  the  target  is  still 
scored  as  a  hit.  However,  the  crosshairs  must  be  within  the 
target  when  you  lase.  For  the  last  block  the  kill  zone  will  be 
set  so  that  you  must  hit  within  the  target  to  be  credited  with  a 
hit. 

Your  goal  is  to  hit  targets  as  rapidly  as  possible. 


Instructions  for  subjects  in  100%  group 

For  the  first  three  blocks  of  trials  and  the  test  block  the 
target  kill  zone  is  set  so  that  a  shot  that  lands  anywhere  within 
the  target  is  scored  as  a  hit.  The  crosshairs  must  be  within  the 
target  when  you  lase. 

Your  goal  is  to  hit  targets  as  rapidly  as  possible. 
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Instructions  for  subjects  in  50%  group 

For  the  first  three  blocks  of  trials  the  target  kill  zone  is  set 
so  that  in  order  for  a  shot  to  be  scored  as  a  hit  the  shot  must 
land  near  the  center  of  the  target.  The  kill  zone  is  actually 
about  1/2  of  the  size  of  the  target  that  you  see.  However,  the 
crosshairs  can  be  anywhere  within  the  target  when  you  lase.  For 
the  last  block  the  kill  zone  will  be  set  so  that  a  shot  landing 
anywhere  within  the  target  will  be  scored  as  a  hit. 

Your  goal  is  to  hit  targets  as  rapidly  as  possible. 


You  should  engage  the  targets  as  quickly  and  as  accurately  as 
possible  at  all  times. 


Are  there  any  questions  regarding  these  instructions? 


TO  SUMMARIZE; 

1.  Manipulate  the  Shutter  and  Thermal  Mode  switches  according 
to  the  experimenter's  instructions  (Normal  or  Thermal). 

2.  Upon  hearing  the  tank  commander,  set  the  Ammunition  Select 
switch. 

3.  Squeeze  palm  levers  and  hold  them  down. 

4.  Move  the  crosshairs  onto  the  center  of  the  target. 

5.  Press  the  lase  button,  continue  tracking. 

6.  After  the  Tank  Commander  says  "Fire"  press  the  fire  button. 

7.  Obseirve  where  the  round  hits,  then  look  away  from  the 
screen. 


